Ensemble Summarization Models to Leverage Performance

Summaries underpin a majority of relevant information needed to quickly make an informed decision from a large corpus of text;Natural Language techniques have been developed to generate these summaries using either ive or extractive methods. Presently, state-of-the-art approaches involve using neural network-based solutions akin to seq2seq, graph2seq, and other encoder-decoder architectures. These models make different contributions to prediction quality. In this paper, we build a model that ensembles two distinct pretraining NLP models to leverage their summarization performance using a TextRank process we constructed. We evaluate our model using the CoronaNet Research Project COVID-19 dataset, which contains how governments responded to the Covid-19 pandemic. We compared the ROUGE scores of the individual models on the test set to our ensemble method. The experiment results show that our proposed ensemble method performs better than using the models individually. © 2022 IEEE.

Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training

Federated learning, which shares the weights of the neural network across clients, is gaining attention in the healthcare sector as it enables training on a large corpus of decentralized data while maintaining data privacy. For example, this enables neural network training for COVID-19 diagnosis on chest X-ray (CXR) images without collecting patient CXR data across multiple hospitals. Unfortunately, the exchange of the weights quickly consumes the network bandwidth if highly expressive network architecture is employed. So-called split learning partially solves this problem by dividing a neural network into a client and a server part, so that the client part of the network takes up less extensive computation resources and bandwidth. However, it is not clear how to find the optimal split without sacrificing the overall network performance. To amalgamate these methods and thereby maximize their distinct strengths, here we show that the Vision Transformer, a recently developed deep learning architecture with straightforward decomposable configuration, is ideally suitable for split learning without sacrificing performance. Even under the non-independent and identically distributed data distribution which emulates a real collaboration between hospitals using CXR datasets from multiple sources, the proposed framework was able to attain performance comparable to data-centralized training. In addition, the proposed framework along with heterogeneous multi-task clients also improves individual task performances including the diagnosis of COVID-19, eliminating the need for sharing large weights with innumerable parameters. Our results affirm the suitability of Transformer for collaborative learning in medical imaging and pave the way forward for future real-world implementations. © 2021 Neural information processing systems foundation. All rights reserved.

Constructing COVID-19 Knowledge Graph from A Large Corpus of Scientific Articles

Creating domain-specific glossaries that are both time-consuming and requires domain expertise. An effective and efficient automatic process will facilitate the glossary generation and its downstream applications for better decision making. In this project, we aim to build a domain-specific glossary from a large text corpus. We form the task as a knowledge graph construction problem with minimum supervision. We adapt both supervised pre-trained models and unsupervised methods for extracting relations for terms appear in the large corpus of scientific articles. We then utilize an off-the-shelf graph database to construct and store the knowledge graph. Furthermore, we develop an interactive Web-based tool for visualizing, exploring and querying the constructed knowledge graph. The project is sourced and funded by AI4DM initiative from the Office of National Intelligence (ONI) and the Defence Science and Technology (DST) Group, Australia. Although the fund requires the usage of a dataset of COVID-19 related literature collection, the solution to be presented in this paper is generic and could be easilt applied to any domain. © 2021 IEEE.

Knowledge Discovery in COVID-19 Research Literature

This paper presents the preliminary results of an ongoing project that analyzes the growing body of scientific research published around the COVID-19 pandemic. In this research, a general-purpose semantic model is used to double annotate a batch of 500 sentences that were manually selected from the CORD-19 corpus. Afterwards, a baseline text-mining pipeline is designed and evaluated via a large batch of 100, 959 sentences. We present a qualitative analysis of the most interesting facts automatically extracted and highlight possible future lines of development. The preliminary results show that general-purpose semantic models are a useful tool for discovering fine-grained knowledge in large corpora of scientific documents. © 2021 Incoma Ltd. All rights reserved.